Exemplar-Based Topic Detection in Twitter Streams

Detecting topics in Twitter streams has been gaining an increasing amount of attention. It can be of great support for communities struck by natural disasters, and could assist companies and political parties understand users’ opinions and needs. Traditional approaches for topic detection focus on representing topics using terms, are negatively affected by length limitation and the lack of context associated with tweets. In this work, we propose an Exemplar-based approach for topic detection, in which detected topics are represented using a few selected tweets. Using exemplar tweets instead of a set of key words allows for an easy interpretation of the meaning of the detected topics. Experimental evaluation on benchmark Twitter datasets shows that the proposed topic detection approach achieves the best term precision. It does this while maintaining good topic recall and running time compared to other approaches. In 2014, active users on Twitter reached more than 645 million and produced approximately 500 million tweets daily.1 From these numbers, users can easily miss important topics. Thus the need for mining this amount of unstructured data became paramount. Topic detection in Twitter is an important mining task that has drawn a lot of attention during the past few years. It is defined as the task of discovering the underlying key topics that occur in a set of tweets. Some of the benefits of this task include: discovering natural disasters as early as possible, helping political parties and companies understand users’ opinions and improving content marketing by better understanding customer needs. Many approaches can be used to detect important topics that occur in a set of documents. However, many challenges arise when traditional approaches are applied on Twitter data. One of the challenges is the scalability of the processing methods to deal with the massive amounts of daily generated tweets. Previous approaches that were developed in literature for topic detection like (Deerwester 1990) and (Aiello et al. 2013) focus on identifying terms that represent the topic regardless of how the terms can be properly connected so that they can be easily interpreted by an individual and regardless of whether or not noisy terms are included in the reCopyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. http://www.statisticbrain.com/twitter-statistics/ trieved set. This motivated us to propose a fast and accurate Exemplar-based approach to detect topics in Twitter based on representing each topic by a single tweet. This Exemplarbased representation alleviates the aforementioned problems and allows for easy understanding of the retrieved topics. The rest of the paper is organized as follows: we start first by discussing some of the related work, then presenting the proposed approach for topic detection. After that, we show the implementation details, experimental results and discussion. Finally, the last section concludes the paper.